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Abstract 

The GIPSY system provides a framework for a distributed multi-tier demand- 
driven evaluation of heterogeneous programs, in which certain tiers can generate de- 
mands, while others can respond to demands to work on them. They are connected 
through a virtual network that can be flexibly reconfigured at run-time. Although 
the demand generator components were originally designed specifically for the educ- 
tive (demand-driven) evaluation of Lucid intensional programs, the GIPSY's run- 
time's flexible framework design enables it to perform the execution of various kinds 
of programs that can be evaluated using the demand-driven computational model. 
Management of the GISPY networks has become a tedious (although scripted) task 
that took manual command-line console to do, which does not scale for large exper- 
iments. Therefore a new component has been designed and developed to allow users 
to represent, visualize, and interactively create, configure and seamlessly manage 
such a network as a graph. Consequently, this work presents a Graphical GMT 
Manager, an interactive graph-based assistant component for the GIPSY network 
creation and configuration management. Besides allowing the management of the 
nodes and tiers (mapped to hosts where store, workers, and generators reside), it 
lets the user to visually control the network parameters and the interconnection be- 
tween computational nodes at run-time. In this paper we motivate and present the 
key features of this newly implemented graph-based component. We give the graph 
representation details, mapping of the graph nodes to tiers, tier groups, and specific 
commands. We provide the requirements and design specification of the tool and 
its implementation. Then we detail and discuss some experimental results. Key- 
words: graph-based management, visualization, GIPSY network, demand-driven 
computation, GUI 
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1 Introduction 

The GIPSY (General Intensional Programming System) project is an ongoing research 
project developed at Concordia University. Its initial goal was to investigate on a general 
solution for the evaluation of programs written in the Lucid intensional programming 
family of languages using a distributed demand-driven evaluation model. In order to 
meet the flexibility goals of the project, the system has been designed using a framework 
approach integrating a Lucid compiler framework, as well as a demand-driven run-time 
system framework. 

In its eductive model of execution, the system assumes the presence of demand gen- 
erators, as well as a demand workers. Each demand generated is paired along with the 
context in which it is made and is uniquely identifled. The demands are migrated using a 
communication node that enables the connection between demand generators and demand 
workers. Through the communication node, any demand worker can pick-up demands, 
compute its resulting value, and send it back to the communication node to be picked up 
by the generator. 

Notably, the framework has demonstrated its flexibility by having the run-time system 
put to use in the demand-driven distributed evaluation of programs not involving the 
Lucid language. The work presented here goes in this direction and makes abstraction 
of the intensional programming aspect of the project and concentrates on the demand- 
driven evaluation of heterogeneous programs. We concentrate on showing how a virtual 
network of demand-driven computational nodes can be represented graphically at run- 
time, enabling the user to map the demand-driven computation nodes over an underlying 
physical network of computers, and to control their execution and connectivity at run- 
time. 

In the current implementation, the node's connectivity is expressed in a set of con- 
flguration flies. Upon starting, a node reads its conflguration flle and establishes its own 
connection according to the information contained in the conflguration flle. The conflg- 
uration can be changed at any time, so that a node can reconflgure its connectivity at 
run-time. 

A manager node, which acts as a supernode, has been implemented to manage a 
GIPSY network. It enables new node(s) to automatically establish connection with it 
and receive commands from it. Therefore, enabling the manager node to remotely change 
the conflguration of any registered node. The virtual network is thus constructed from the 
interconnection of the generators, workers, communication, and manager nodes, the later 
being able to establish the connectivity between the three flrst ones. All communication 
between nodes, including commands exchanged for conflguration changes, are using the 
same demand-driven communication mode. 

The rest of this paper is organized as follows: Section 2 gives on overview of the 
GIPSY Framework and its multi-tier architecture, the GIPSY run-time system and flnally 
discusses the related work. Section 3 summarizes the objectives of this work. Section 4 
presents how currently the GIPSY run-time system is being managed, discusses the design 
and implementation of the proposed solution and evaluates the results of some conducted 
experiments. Then, Section 5 concludes the paper and points out new research direction 
planned as future work. 
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2 Background 

2.1 GIPSY Framework 

The GIPSY run-time system is a distributed multi-tier and demand-driven framework. It 
mainly consists of a set of loosely coupled software components enabling the evaluation 
of programs in a distributed demand-driven manner. The run-time system is composed 
of the following basic entities [17]: (a) A GIPSY tier is an abstract and generic entity. 
Each tier instance is a separate thread (one or more) that runs within a registered pro- 
cess, namely (GIPSY node), and represents a computational unit that contribute to the 
distributed computation. Tiers cooperate in a demand-driven mode of computation; (b) 
A GIPSY node is a registered process that hosts one or more GIPSY tier instances be- 
longing to different GIPSY instance(s). Node registration is done through a manager tier 
called the GIPSY Manager Tier (GMT). More specifically, a node is a computer running 
a GIPSYNode process; (c) A GIPSY instance is a group of tier instances collaborating 
together to achieve program execution. It can be considered as a set of interconnected 
GIPSY tier instances hosted/deployed on one or more GIPSY nodes executing GIPSY 
programs by sharing their respective resources. A GIPSY instance can be executed across 
different GIPSY nodes. Moreover, as shown in Figure 1, a GIPSY network is designed as 
an overlay network where network nodes, GIPSY tiers, are organized in a cluster called 
GIPSY instance. A GIPSY tier can be seen as a virtual network node and hosted on a 
GIPSY node. In such a network, the mapping between a GIPSY node and a physical 
node is made upon starting and registering the node through the GMT. 

2.1.1 Multi-Tier Architecture 

In [17], a distributed multi-tier architecture has been defined and adopted in the imple- 
mentation of GIPSY run-time system. The architecture inherits some of the peer-to-peer 
network architecture principles, e.g (1) no single-point of failure: any tier or node can 
fail without fatally affecting the system; (2) nodes and tiers can seamlessly join/leave the 
network by adding/removing them on the fiy as computation is happening; (3) demands 
are propagated without knowing where they will be processed or stored; (4) available 
nodes and tiers can be affected at run-time to the execution of any GIPSY program while 
other nodes and tiers could be computing demands for different programs. The multi-tier 
architecture is composed of four distinct tiers: (a) a Demand Store Tier (DST) that acts 
as a middleware between tiers in order to migrate demands, provides persistent storage 
of demands and their resulting values (demands caching), and exposes Transport Agents 
(TAs) used by other tiers to connect to the DST; (b) a Demand Generator Tier (DGT) 
that generates demands according to the declarations and resources stored in the GEER 
generated for the program being evaluated. The DGT maintains a local demand pro- 
cessing dictionary pool that contain the definitions required to formulate demands; (c) 
a Demand Worker Tier (DWT) which processes demands by executing method defined 
in such a dictionary. The DWT connects to the DST, retrieves pending demands and 
returns back the computed demands to the DST; (d) a General Manager Tier (GMT) 
(see Figure 1), as its name implies, locally and remotely controls and monitors other tiers 
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(DGT, DWT and DST) by exchanging system demands. Furthermore, the GMT can 
register new nodes, move tier instances from one node to another, or aUocate/deaUocate 
tier instance from/on a registered node. 




Figure 1: Example of a GIPSY Nodes Network [17] 



2.1.2 Demand Types and States 

A demand is a run-time request asking for a value of certain identifier defined in a GIPSY 
program. Demands are migrated to other tiers using the DST. There are three types of 
demands: (a) intensional demands, which are generated for the evaluation of a GIPSY 
program by a generator. GIPSY programs are written in a declarative style, where an 
identifier is defined as an expression using other identifiers defined in a multidimensional 
context space [17]. All demands for GIPSY identifiers contain the context in which they 
are made, and their evaluation depend on this context. The GIPSY program also uses 
an algebra of procedures that can be called during evaluation, which are called at run- 
time and become procedural demands; (b) procedural demands which are generated by the 
DGT when it encounters a procedural function call during the GIPSY program evaluation. 
Procedural demands are processed by the DWT; (c) system demands, in turn, are issued 
by the GMT for run-time management purposes and include demands for monitoring and 
controlling tiers at run-time. It is worth mentioning that system demands are requests for 
managerial tasks e.g. demand for node registration, tier allocation and deallocation. In 
contrast, intensional demands and procedural demands are computational demands, i.e. 
demands that are generated during the evaluation process of a specific GIPSY program. 

In the GIPSY environment, each demand has a state and demand states are used to 
manage and propagate demands. The state transitions are managed by the demand store 
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tier DST, who is responsible for demand migration between generator and worker tiers. 
We distinguish three possible states as follows: (a) Pending: a pending demand is a 
demand that has been issued by a tier to the demand store and not yet picked by another 
tier for further processing. Pending demands are sent to generators and workers when 
they notify their availability for processing [17]; (b) Processing: a processing demand 
is a demand that has been grabbed by a tier from the demand store and its evaluation 
still being processed. This state is assigned by the demand store in order to make sure 
that the same demand is grabbed by only one tier for processing. When a tier goes out 
of service, all its associated processing demands are put back to the pending state by 
the demand store, ensuring a fail-safe behavior [17]; (c) Computed: indicates that a 
demand has been computed. When a tier grabs a demand and is finished computing its 
corresponding value, it sends back the result to the demand store, that stores it in place 
of the initial demand and marks it as computed. Any further demands generation with 
the same context will result in the store to directly respond with the resulting value, thus 
saving computation time [17]. 

2.2 Graphical GMT Tool Support for the GIPSY Run-time Sys- 
tem 

Here we provide some details how the graph-based tool we developed assists with the 
automation of the startup sequence and management tasks of the system. 

Configuration. The tier instantiation process has a flexible design and has been imple- 
mented using Java Reflection [5] and the Factory design pattern [4] . It uses a conflguration- 
based system to instantiate tiers on the fly. Generic conflguration instances are stored in 
flies and their settings can be easily updated and tailored to a speciflc tier's implementa- 
tion requirements. Conflgurations contain a set of key-value pairs where the key denotes 
the name of the conflguration property while the values could be anything from a service 
name, port number, IP address, etc. Such a conflguration system eliminates the need of 
writing or adapting source code to reflect a speciflc tier conflguration. The properties 
stored in the Configuration object determine the tier class to instantiate and consist 
of different settings interpreted by the tier implementation class. Upon receiving a tier 
instantiation request, the TierFactory inspects the conflguration instance to determine 
which tier implementation class to instantiate using Java Reflection [5]. 

Bootstrapping, as mentioned earlier, a GIPSY network consists of a set of intercon- 
nected GIPSY nodes each hosting GIPSY tiers mapped to physical machines where the 
GIPSY run-time is deployed. Such a network is managed by a GIPSY manager tier (GMT) 
that enables nodes registration to the network and tier allocation on the registered nodes. 
The bootstrap process of the GIPSY manager tier starts a registration demand store that 
is used solely for the exchange of system demands with the nodes and tiers allocated in 
the GIPSY network managed by this manager tier. Thus, system demands and compu- 
tational demands are exchanged using different communication channels. Any computer 
deploying the GIPSY node run-time system can send a registration request to the man- 
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ager tier, enabling this manager tier to remotely connect and control the allocation of 
various tiers on the registered nodes. After tiers are allocated to the registered nodes, the 
manager tier can connect the different tiers together, and eventually instruct a genera- 
tor to start the demand-driven evaluation of a GIPSY program. Even after execution is 
started, the manager tier can accept new nodes registrations, or allocate/deallocate new 
tiers on any registered node that it manages and make newly allocated tiers to contribute 
to a program's evaluation on the fly [8, 7]. 

GIPSY Node Registration. When a node wants to join the network, first, a GIPSY 
node issues a request to the GMT expressed as NodeRegistration system demand having 
pending as state. Upon receiving a NodeRegistration demand, the GMT assigns a DST 
to the GIPSY node who issued the request. Afterward, the GMT saves the node regis- 
tration information in a GMTInf oKeeper object and sends back a RegistrationResult 
demand having computed as state and containing the DST information and the assigned 
node ID. Finally, the GIPSY node processes the result and uses the information con- 
tained in the demand to establish a connection to the assigned DST. Establishing such 
connection creates a communication channel for further exchange of system demands. 

Tier Allocation. Tiers are allocated inside a previously registered GIPSY node. The 
process of tier allocation is performed through the operation of the GMT using a pair of 
system demands: TierAllocationRequest and TierAllocationResult. Both demands 
share the same demand signature but have different states: pending and computed respec- 
tively. The following information needs to be specified in the TierAllocationRequest 
demand: the node identifier of the GIPSY node where the tiers to be allocated, the type 
of the tier and how many tier instances are to be allocated. When the allocation process 
is completed a TierAllocationResult demand is triggered and contains a set of tier 
registrations. Each tier registration contains information such as the tier identifier, which 
is internally assigned by the GIPSY node. 

Tier Deallocation. Tier deallocation consists of removing previously allocated tiers. 
Similarly to the case of the tier allocation process, two system demands 
TierDeallocationRequest and TierDeallocationResult are issued by the GMT to 
deallocate tiers upon user's request. The type and the ID of the tier to be deallocated 
are embedded in a TierDeallocationRequest and sent to the GIPSYNode process to 
deallocate the tier specified. 

2.3 Related Work 

Related work by several researchers on visualization of load balancing, configuration, 
formal systems for diagrammatic modeling and visual languages and the corresponding 
graph systems are presented in [20, 19, 1, 2, 10]. They all define key concepts that are 
relevant to our visualization mechanisms within GIPSY and its corresponding General 
Manager Tier [7]. 
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3 Objectives 

The GIPSY framework has been designed in a modular manner but has a lot of config- 
urable components; hence, the need of an automation solution for configuring and manag- 
ing GIPSY deployment components is crucial. Moreover, prior to this work, the run-time 
system was managed using primarily a command-line interface. The project should pro- 
vide an integrated tool that allows the user to: create a GIPSY network and configure its 
components (GIPSY instances, tiers and nodes); save/load a GIPSY network configura- 
tion; start/stop GIPSY nodes and register them with the GMT, and allocate/deallocate 
GIPSY tiers; dynamically visualize GIPSY nodes and tiers and inspect/change their prop- 
erties at run-time; change tiers connectivity at run-time; increase the usability of GIPSY 
run-time system as a whole; provide means and semantics for scheduling, validation, and 
visual mapping to Lucid programs. The GMT is the central element of our system from 
the user's perspective. It enables to handle the managerial tasks related to the configura- 
tion and functioning of a GIPSY network. The proposed solution should be transparent 
and efficient enough in order to enhance the system usability, ffexibility, and end-users 
experience, while maintaining the structure for run-time analysis and scheduling. 

4 Solution 

4.1 Overview 

The solution presented in this paper is a graph-based graphical user interface that provides 
a set of user interfaces enabling the users to directly interact with the distributed GIPSY 
run-time system. The main objectives (cf. Section 3) of this work consist of increasing 
the usability of the run-time system and enabling the user to have full control over the 
GIPSY network with a minimum of detailed manual intervention. It should be noted 
that, prior to this work, all the managerial and configuration tasks needed to bootstrap 
a GIPSY network required the user to manually execute shear number of commands and 
scripts. In this work, we designed the graphical GMT component that aims at allowing 
the user to manage and operate the entire GIPSY network seamlessly by translating 
simple graphical user interactions into complex message passing between the underlying 
deployment components. Our solution enables the user to easily create, configure, and 
control a GIPSY network through a graph-based interface. GIPSY tiers are illustrated as 
connected graph nodes. Tiers' properties are read from files and stored as Configuration 
objects embedded in the graph nodes. We use graph element shapes to differentiate 
GIPSY instances and colors to differentiate GIPSY nodes. When the user adds a new 
tier to the network graph, the color assigned to the tier is associated to the node the tier 
is assigned to. 

According to the GIPSY multi-tier architecture, the DWT, DGT and DST expose 
software interfaces to be used for their mutual interactions. Since the GMT plays a key 
role in the GIPSY network management, it provides a handy mechanism for starting 
and stopping nodes, and allocating and deallocating tiers. In the current run-time im- 
plementation, the interaction with the GMT is command-based and is done through a 
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command-line console UI, with which the user manually bootstraps and controls the nodes 
and tiers by entering commands the corresponding. Additionally, a set of configuration 
files with the appropriate settings and properties for each tier type are needed. Before 
performing any node or tier startup or registration, we assume that a set of configuration 
files with appropriate settings and properties for nodes each tier type have been created. 
Typically, in order to start a network, the following sequence of steps should normally be 
performed: 1. At first, a GIPSY node process should be created; that prompts the user to 
start a GMT tier. When a GMT is started, the GIPSY node is automatically registered 
and a registration DST is allocated [7] . The registration DST enables the GMT to receive 
system demands for further node and tier allocations. This is the initial bootstrapping 
process that enables all further operations on a GIPSY network. 2. At any time the user 
can expand the network by adding an additional node locally on the computer where the 
GMT is executing (and tiers on a remote computer). Upon successful node creation, the 
user is prompted to register the node to an existing GMT using the register command. 
3. Then, on any registered node, DSTs are started to allow the propagation of demands 
between generators and workers, DGTs are registered to generate demands, and DWTs 
are registered to process the generated demands. 

Based on this discussion, the following is a typical list of example commands that are 
used to interact with the run-time system in order to setup a manual GIPSY network: 

1. start GMT GMTConf igFile . conf ig 

This command starts the bootstrap process explained above, where GMT is the 
type of the tier and GMTConf igFile . conf ig is the configuration file that contains 
the settings and properties needed to instantiate a GMT tier instance. 

2. allocate NodelD TierType TierTypeConf igFile DSTIndex HowMany 

This command allocates a DGT or a DWT. NodelD is the numeric ID of the node 
where the tier should be allocated, TierType is the type of the tier ([DGT, DWT]), 
DSTIndex is the index of the DST, to which the tier in question should connect to 
and the TierTypeConfigFile is the tier-specific configuration file to use. 

3. allocate NodelD DST DSTConf igFile . conf ig HowMany 

This command sends a request to the GMT with the node ID where a DST instance 
will be allocated, how many DST instances are needed, and a DST configuration 
file name. 

4. deallocate NodelD TierType TierlDl TierID2 TierlDn 

This command issues a demand to be processed by the GMT to deallocate tiers. 
TierType is the type of the tierID[l..n] is tier instances IDs to deallocate in a node 
specified by its ID. 

The GIPSY network configuration process requires the user not only to know all the 
commands and their exact syntax, but requires to keep track of the IDs of the nodes 
and tiers. It also requires the user to manually edit the related configuration files. The 
configuration files contain many configuration elements that are not of importance in the 
node/tier management process, thus leading to confusion and possible mistakes. Our 
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newly designed graphical GMT assistant rather allows the user to abstractly manipulate 
icons and use menu options to effectuate these operations. These GUI operations initiated 
by the user are then translated by the graphical GMT into the commands similar to the 
presented earlier. As for the changes to the configuration files, the user is presented 
through the GUI with only the configuration elements that are relevant in the context 
of use, thus reducing the information load on the user and reducing the possibility of 
configuration mistakes. Listing 1 shows the content of a configuration file for a DGT [7]. 
It provides configuration information such as to which class implementation to instantiate, 
the number of instances to be created, the mode of communication to use, and a maximum 
number of demands that can be generated. These configuration parameters are read 
during startup and will determine the behavior of the generator. 



# Which implementation of the DGT class to instantiate . 

gipsy . GEE . mult it ier . wrapper . impl = gipsy . tests . GEE. simulator . DGTSimulator 

gipsy . GEE . mult it i er . DGT . DemandDispatcher . impl = gipsy . GEE . I DP . DemandGenerator . jini . rmi . 
JiniDemandDispatcher 

# Concurrent asynchronously 

# 1 User - controlled asynchronously 

# 2 Response time tester: synchronously 

# 3 Space - scalability tester, 
gipsy . tests . GEE . simulator . mode =2 

gipsy . tests . GEE . simulator . tester . parameter=l 

# Number of instances to be created, 
gipsy . tests . GEE . simulator . tester . number =2 

# Number of maximum demands . 

gipsy . tests . GEE . simulator . demand . pay load =3 2 

Listing 1: A Sample of DGT Configuration File 



4.2 Design and Implementation 

Our implementation relies on the graph-based visualization to illustrate a GIPSY network. 
We represent a GIPSY tiers network as interconnected graph nodes where each such node 
contains data/properties used in tier-to-tier communication configuration. Such proper- 
ties are assigned and configured by the user when creating a GIPSY network. The GMT 
GUI was implemented using the Java JFC/Swing library. The GIPSY's Configuration 
class is used to store diflFerent components' configuration. We have selected the Java Uni- 
versal Network/Graph (JUNG) library to implement the visualization of the management 
aspect of GIPSY nodes [9, 15]. JUNG is an open-source library for modeling data that 
can be represented as a graph or a network. JUNG provides many visualization features 
that can be changed at runtime such as node color, shape, and size. Thus, graph nodes 
can be grouped together, which enables us to diflFerentiate the nodes by their tier type 
(DST, GMT, or DWT). Through JUNG, GIPSY nodes are configured while creating a 
connected graph of nodes and to visualize and manage their activities to alleviate manual 
complexity of such operations. The GMT GUI addresses the need of the automation of 
the managerial tasks of the GIPSY run-time system and the configuration of resources. 

The implemented features are: 1. create a new GIPSY network as a graph; 2. save/load 
a pre-configured GIPSY network to/from files; 3. start, register and stop GIPSY nodes by 
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maintaining a color-differentiated list of nodes with their related commands and configu- 
rations available in a context menu; 4. allocate and deallocate DSTs, DGTs and DWTs 
by manipulating icons and context menus; 5. start/stop the demand-driven evaluation 
process on a DGT trough a contextual menu accessed on its icon. 

The process of node registration and tier allocation has been embedded into our tool, 
and only the most relevant configuration information is shown to the user. Graphical 
objects representing GIPSY nodes encapsulate their related commands and hold the nec- 
essary properties for user inspection or change at setup or run-time. As for GIPSY tier 
graphical objects, in addition to allocate and deallocate commands, these objects provide 
a drag-and-drop mechanism used to change the connectivity between tiers on the fiy at 
run-time (see Figure 2(a)). When a new tier or a GIPSY node is added to the network, 
it is automatically pre-configured and associated with a configuration file with the prop- 
erties entered by the user (see Figure 2(b)). The GMT GUI is arranged in a tabbed form 
and provides two main distinct editor and operator views. 

The network graph editor and resource configuration allow to create a GIPSY network 
or load an existing one. As shown in Figure 2(b), GIPSY instances and nodes are arranged 
in two lists while GIPSY tiers in a graph illustrating interconnected tiers. Instances, 
nodes and tiers can be easily added and configured separately. The configuration process 
is completely automated using dialog boxes allowing the user to fill in the configuration 
properties of each entity. All data entered is validated allowing only valid values to 
be accepted. In this editor, two GIPSY tiers could be connected together and their 
configuration commands automatically generated by drawing a line to connect two graph 
nodes. 

The GMT operator lists context-menu-enabled GIPSY nodes allowing the user to start 
or stop GIPSY nodes and register them with the GMT by simple mouse clicks. As illus- 
trated in Figure 2(a), GIPSY tiers are shown as connected graph nodes. Tiers belonging 
to the same GIPSY instance are assigned the same shape. The tier's color determines on 
which GIPSY node a given tier is hosted on. Moreover, inspection and visualization of 
any element's properties is possible at run-time. This enables the user, for instance, to 
know which GIPSY tier is residing on which GIPSY node. The run-time system activities 
such as the output of GMT, GIPSY nodes and GIPSY tiers, errors, and log messages 
are displayed in a separate distinct views. This provides better failure traceability and 
errors troubleshooting while, at the same time, providing useful information related to 
the overall computation process. 

In Figure 3(a) is a set of JUNG-interfaced classes we produced to integrate with 
and visually represent, load, save, and manage GIPSY networks while in Figure 3(b) 
we detail the data structures used to internally represent the network graphs and map 
them to the GIPSY objects and the action items associated with them. NodeConnection 
is a semantically central data structure that links graph elements representing GIPSY 
tiers (the instances of GIPSYTier classes). These connections and tier properties are 
the actual representation of the graphs that are saved to and loader from a name:value 
paired configuration files (e.g. see Appendix B) by GraphDataManager. A collection 
of NodeConnections is managed by the GraphViewer, and both NodeConnection and 
GIPSYTier have action items attached to them that send the aforementioned GIPSY 
commands to actually do the work via the visual NodeMenu and TierMenu. Every tier has 
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Figure 2: GMT Operator and Network Graph Editor Views 
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Figure 3: UML Class Diagrams of the Graph- Related Design Aspects 



4.3 Results and Discussion 

The results are encouraging since they demonstrate the ability of the proposed solution 
to assist in automation of some management functions GIPSY run-time system. We have 
first tested with the simulator [18, 7] which allows to generate different types of demands 
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to be computed. We have then performed some usabihty testing with another tool, that 
was recently adapted to be distributively executed over GIPSY - MARFCAT. It is one 
of the realistic long-running distributed pattern recognition computation processes test 
cases (e.g. MARF's pattern recognition pipeline [11] with very large data sets over GIPSY 
for the static code analysis application for vulnerabilities and weaknesses detection and 
malware classification [12, 14]. MARFCAT was made to run completely over GIPSY 
separating the heavy and light work logic across the generator and worker tiers. The tool 
properly starts up all the indicated components, the network of which were created and 
the configurations loaded, begins the computation, logs the output to the console, and 
while computation proceeds, the tier state is properly refiected visually. 

While us we were the only users of the proposed PoC tool thus far, by making it 
public and released along with the GIPSY system and via the demonstration of the tool 
on the GIPSY simulator and MARFCAT, we hope to gather larger feedback on the tool 
to improve its usability further while weeding out known bugs. 

In Appendix A we summarize a complete demo procedure/manual for a creation of a 
specific application to run over the GIPSY network for demonstration purposes. 

Platforms Tested. We tested the tool in various operating system platforms to ensure 
the portability is maintained: Windows XP SP3 32-bit, Windows 7 32-bit and 64-bit; 
Scientific Linux 6.2 32-bit and 64-bit, and Ubuntu Linux 1L03 32-bit under VMware; 
MacOS X 10.5 32-bit and 64-bit; Oracle JDK 6 and 7; OpenJDK 6; Apple JDK 6. 

Implications. The implications of this work are multifold. First, the usability and 
management aspects of the multi-tier GIPSY network are of obvious mention. Addition- 
ally, having the network represented and managed as a graph allows for further reasoning 
and automatic scheduling [6] and load-balancing of such a network through the graph 
analysis. Thirdly, since Lucid is a data-flow language and was shown to have one-to-one 
correspondence with the data-flow graphs (DFGs) [16, 3, 13], the tool opens up more 
possibilities for diagrammatic programming and program-graph-execution-network trans- 
lation model for detailed analysis and veriflcation of Lucid-based programs with the added 
visualization beneflt. 

5 Conclusion and Future Work 

We have presented a graph-based GUI implementation for the simpliflcation of the man- 
agement of the GIPSY run-time system components. The presented tool is proving to be 
an effective solution assisting with management automation of GIPSY software artifacts 
distributed across multiple physical machines forming an overlay network. Our solution 
relies on graph-based programming and visualization to represent a GIPSY network. Each 
graph node represents a GIPSY tier and is pre-conflgured and loaded with the information 
needed at run-time. A GIPSY network can be created, conflgured and saved to a flle. The 
user can establish a connection between pairs of GIPSY tiers by drawing a line to connect 
two graph nodes. A GIPSY network can be easily bootstrapped and managed on the 
fly. Many demand generators and workers can be allocated as computation is happening. 
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While aiming at increasing the usability of the run-time system, our solution allows the 
user to seamlessly inspect the status and properties of GIPSY nodes and GIPSY tiers at 
run-time. 

The work presented in this paper is to be extended, thus, additional features and 
improvements are planned. Future work includes a better semantic definitions of the 
graph manipulation actions, so that any operation on a graph can be translated more 
easily into the underlying system's commands and be verifiable. We plan to add observers 
to any graph element, enabling for example to click on a graph link to observe the demands 
fiowing across this link at run-time. Among the planned future works is the continual 
extension of the current design to support more problem-specific tiers like MARFGAT, 
e.g. genome sequence alignment, and similar computation problems that need a lot of 
manual pre-setup to run. We further plan to allow intra-tool (peer) communication to 
further allow start up nodes on remote computers and not only tiers. Additionally, expose 
an OpenGL-to-Java remote interface to allow connecting to the tool from any OpenGL- 
enabled systems remotely, including mobile devices based on iOS and Android. 
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A Mini Demo User Manual / "Demo Paper'' 

In this section we give a brief overview on how to use the graphical PoC tool to create 
and manage a GIPSY network. We explain how to create and configure a network, how 
the network is saved/loaded to/from a file, and finally, how the network is being managed 
in the GMT Operator view. 

This section is intended for the demonstration of the tool at the conference with a 
mini-user manual instructions and operational description. We will show how to create 
and start a complete GIPSY network for the specific MARFCAT application and perform 
a complete run of it. We will release the PoC tools, the application, and the source code 
for the audience and community at large as well. 

A.l Using the Graph editor 

The graph editor is used to create a new GIPSY network or to edit an existing one. 

I. Creating /Editing a GIPSY instance 

Figure 4(a) and Figure 4(b) show how to create a GIPSY instance and edit its 
information. By clicking on the add button, the user enters a name for the new 
GIPSY instance to create and clicks save. Double-clicking on an instance in the list 
of instances allows to edit the instance name in an appropriate dialog. 

II. Creating /Editing a CIPSY Node 

To create a new GIPSY node, click on the "Add" button Figure 4(c), which pops 
up a dialog to fill in the new node's properties such as the node name, IP address 
and color, see Figure 5(a). Upon clicking "Save", the new node will be added to the 
list as shown in Figure 5(b). To edit an existing node's properties, double-click on 
an item in the node list and a editing dialog will pop up Figure 5(c). 



■a GIPSY - Graph-Operated GMT 




(a) Creating new GIPSY In- 
stance 



■a GIPSY - Graph-Operated GMT 



□ |e|a|©l<:^l^l@ 



]rpaph Editor | 



Change Mouse ^^ 



'ii Editing GIPSY Instance: INSTANCEl 



Instance Name: mSTANCEl| 



(b) Editing GIPSY Instance 



GIPSY Nodes; 



Add GIPSY Node 



Messages Errors 



(c) Add GIPSY Node 



Figure 4: Adding a GIPSY Instance and a GIPSY Node 
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"a Create New GIPSY Node 

GIPSY Node Properties; 

Node Name: Node! 

Node ID: |o 



IP Address: I192.16S.2.11 



Selected Color; |^ 



(a) Create New GIPSY Node 




(b) New GIPSY Node 
Added 



^ Editing GIPSY Node: NODEl 

GIPSY Node Properties: 

Node Name: |nODE1| 
Mode ID: [o~ 



P Address: 192.163.2.11 



Color; Orange 
Selected Color; | 



(c) Editing GIPSY Node 



Figure 5: Creating and Editing a GIPSY Node 



III. Creating /Editing a GIPSY tier 

While editing, a GIPSY tier is represented as a red graph node. To create a GIPSY 
tier, the user must double-click on the highlighted area as shown in Figure 6(a). 
Then, the tier properties such as the tier name, how many instances to create, 
GIPSY instance to which the tier belongs to, GIPSY node on which the tier will be 
allocated, and finally a configuration file should be specified, cf. Figure 6. To edit a 
given tier's properties, right-click on a graph node and select "Edit Tier Properties" , 
see Figure 7(b). 




Figure 6: Creating and Configuring a GIPSY Tier 



A. 2 Saving/Loading a graph 

To save/load a network to/form a file, use the save/load buttons located in the toolbar. 
Figure 8(a) and Figure 8. 
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A.3 Using the GMT Operator 

This feature is implemented in the "GMT Operator" tab and enabled upon loading a 
valid saved network graph. After loading is complete, the graph nodes (GIPSY Tiers) 
have the same color as the GIPSY node they belong to. Tiers' shapes, as mentioned early 
in this paper, indicate what GIPSY Tier belongs to what GIPSY Instance. 



^ GIPSY - Graph-Operated GMT 




File 




|iGiiie|y|oi^l^l© 




GMT Operator 1 Grpaph Editor | 






1 GIPSY Instances: 


Change Mouse mode 









® GIPSY - Graph-Operated GMT 



fillip I 



I gmtFaufTiers.corfig 
f Test2.con-fig 




(a) Saving/ Loading Graph (b) Load Graph (c) Loaded Graph Example 

Figure 8: Saving and Loading GIPSY network Graph 



I. Starting/Stopping a GIPSY node 

To start a GIPSY node, right-click on a item in the nodes list and select "Start 
Node", Figure 9(a). 

After starting the first GIPSY Node, the actions taken are logged in the log console 
in "Messages" tab. 

When the GMT is first started, a new tab is added to the log console where its 
activities are logged, see Figure 9. 
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^ GIPSY - Graph-Operated GMT 
File 



start Instance 



GMT Operator | Grpaph Editor| 
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Start Node 
Register Node 
STOP Node 
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MANAGER 

DST 

DST2 

DSTl 

DST3 
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Messages pErrors | GMT l] 



loading Graph: 

Opeiming : save dgraphs/gml: Four! i ers ■ config 



GIPSYGMTOperator] Starting node: NODEO 
JiniDSTWrapper] Allocating new Jini DST . . . 
JiniDSTWrapper] Jini DST started, PID = 5772 
GIPSYHodel Node started! 

GIPSYNodel Waiting far a new system demand . . 



Messages | Errors] GMT 1 1 



-Node registered! 
-Node 1 registered! 



(a) Starting Node (b) Starting Node Messages (c) Node Registered 

Figure 9: Saving and Loading GIPSY network Graph 



11. Allocation/ Deallocating GIPSY tier 

To aUocate or deaUocate a GIPSY Tier, right-chck on a graph node and select the 
appropriate action, Figure 10(a). The messages and action triggered by the alloca- 
tion/deallocation process are logged and showed in the console tab. Figure 10(b), 
Figure 11. 




[GIPSYNode] Waiting for a new systerr. derr.and . . . 
[GIPSYNode] Received a DSTAllo cat ionRe quest 
[ Jini E?ST Wrapper] Allocating new Jini DST . . . 
[JiniDSTWrapper] Jini DST started, PID = 5760 
[GIPSYNode] DSTO was allocated and started! 

[GlPSYNode] Sending TierAllocationResult enclosing tier registrations 
[GIPSYNode] Waiting for a new system demand . . . 



(a) Start Tier (b) Allocation/Startup Console Messages 

Figure 10: Starting a Selected Tier 



B Stored Graph Example 

In the example below is a concrete on-disk representation of the GIPSY network graph 
from marf cat4Some . conf ig that can be stored and retrieved and executed instantiating 
the designed configuration and its connectivity for the MARFCAT test case with the 
corresponding graph in Figure 12. 

#Graph data 

#Mon Jun 25 23:04:50 EDT 2012 



19 



Graph-Based Tool To Manage GIPSY Networks 



Rabah, Mokhov, Paquet 




(a) Allocation of DST 



GIPSYHQde] DSTO deallocated! 

GlPSYHode] Sending deallocation result . . . 

GlPSYNode] TierDeallocationEesult sent! 

GIPSYNode] Waiting for a new system demand . 



(b) Deallocation after a System 
Demand 



Messages I Errors I GMT 1 1 



= 1 registered! 

seating 1 DST(3) In Node 1 using C:\n3e 
r allocation finished! 



(c) Deallocation Completed 



Figure 11: Allocation and Deallocation of a DST 



GIPSY - Graph-Operated GMT 




Figure 12: MARFCAT GIPSY network Graph 



gipsy .tier . posx . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=405 
gipsy .tier . isgmt . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=f alse 

gipsy . tier . ID . 1523569d-25aa-44a4-97d5-370b93790333=1523569d-25aa-44a4-97d5-370b93790333 
gipsy .tier . posx . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=409 

gipsy . connection . to . 7802414d-437f -4382-9162-15d9d8e735c3=77f d90ec-e49e-4977-a55b-0690e2e0cd08 

gipsy .tier . howmany . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=l 

gipsy .tier . isgmt . c341ab91-161e-419d-a7e4-6206e75ec097=f alse 

gipsy .tier . howmany . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=l 

gipsy . tier . node . 1523569d-25aa-44a4-97d5-370b93790333=l 

gipsy .tier . posx . c341ab91-161e-419d-a7e4-6206e75ec097=74 

gipsy .tier . howmany . c341ab91-161e-419d-a7e4-6206e75ec097=l 

Connection_key.0dd7156f-3b40-4df7-b9a9-dlb44eeb4da0=0dd7156f-3b40-4df7-b9a9-dlb44eeb4da0 
gipsy. node. name. 3=WDRKER COIL 
gipsy. node. name. 2=WDRKER NETD 

gipsy .tier . posy . 1523569d-25aa-44a4-97d5-370b93790333=55 
gipsy . node . name . 1=W0RKER GLEAM 

gipsy .tier . isgmt . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=f alse 
gipsy . node . name . 0=MAIN 

gipsy .tier . posx . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=75 

gipsy . connection . to . 0dd7156f -3b40-4df 7-b9a9-dlb44eeb4da0=77f d90ec-e49e-4977-a55b-0690e2e0cd08 
gipsy .tier . howmany . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=l 
gipsy .tier . node . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=0 

gipsy . connection . id . leeea2b5-ec50-4990-a7f e-be48bb5b693e=leeea2b5-ec50-4990-a7f e-be48bb5b693e 
gipsy .tier . t iertype . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=DWT 
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gipsy .tier . posy . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=53 

gipsy . tier . ID . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=29310d89-f d7a-498c-8a47-f 80d3b5b9865 
gipsy . tier . isgmt . 1523569d-25aa-44a4-97d5-370b93790333=f alse 

gipsy. connection. id. 6a9el23b-c375-4786-9dl5-12faeee54c59=6a9el23b-c375-4786-9dl5-12faeee54c59 
gipsy . t ier . posx . 1523569d-25aa-44a4-97d5-370b93790333=409 

gipsy .tier . instance . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=8f 5d7614-49e9-4c8a-87f e-c8032a5c9009 
gipsy .tier . howmany . 1523569d-25aa-44a4-97d5-370b93790333=l 

gipsy . connection . name . 6a9el23b-c375-4786-9dl5-12f aeee54c59= [MARFCAT DGT,MAIN DST] 
gipsy .tier . t iertype . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=DWT 
gipsy . tier . tiertype . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=DST 

gipsy. connection. id. 36485580-5acf-4cl2-9110-47c60e23ddc9=36485580-5acf-4cl2-9110-47c60e23ddc9 
gipsy. connect ion. name. leeea2b5-ec50-4990-a7fe-be48bb5b693e= [MARFCAT DWT COIL, MAIN DST] 
gipsy .tier . tiertype . c341ab91-161e-419d-a7e4-6206e75ec097=DGT 
gipsy .tier . isgmt . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=true 

gipsy. connection. name. 36485580-5acf-4cl2-9110-47c60e23ddc9=[MARFCAT DWT NETO,MAIN DST] 
gipsy . t ier . posx . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=74 

gipsy . tier . instance . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=8f 5d7614-49e9-4c8a-87f e-c8032a5c9009 
gipsy. connection. from. 7802414d-437f-4382-9162-15d9d8e735c3=29310d89-fd7a-498c-8a47-f80d3b5b9865 
gipsy .tier . howmany . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=l 

connection_key.6a9el23b-c375-4786-9dl5-12faeee54c59=6a9el23b-c375-4786-9dl5-12faeee54c59 
gipsy .tier . name . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=MARFCAT DWT COIL 

gipsy .tier . instance . c341ab91-161e-419d-a7e4-6206e75ec097=8f 5d7614-49e9-4c8a-87f e-c8032a5c9009 
gipsy . tier . tiertype . 1523569d-25aa-44a4-97d5-370b93790333=DWT 

gipsy. tier. configfile.a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=bin/multitier/DSTProfiles/marfcatDWT.config 

connect ion_key. leeea2b5-ec50-4990-a7fe-be48bb5b693e=leeea2b5-ec50-4990-a7fe-be48bb5b693e 

tier_key.a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6 

gipsy. connection. to. 6a9el23b-c375-4786-9dl5-12faeee54c59=77fd90ec-e49e-4977-a55b-0690e2e0cd08 

tier_key.89e6605e-5c81-4d95-bf51-8e7576c8dcd7=89e6605e-5c81-4d95-bf51-8e7576c8dcd7 

instance_key.8f5d7614-49e9-4c8a-87fe-c8032a5c9009=8f5d7614-49e9-4c8a-87fe-c8032a5c9009 

Connection_key.36485580-5acf-4cl2-9110-47c60e23ddc9=36485580-5acf-4cl2-9110-47c60e23ddc9 

gipsy .tier . instance . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=8f 5d7614-49e9-4c8a-87f e-c8032a5c9009 

gipsy. connection. to. leeea2b5-ec50-4990-a7fe-be48bb5b693e=77fd90ec-e49e-4977-a55b-0690e2e0cd08 

gipsy. connection. from. 0dd7156f-3b40-4df7-b9a9-dlb44eeb4da0=1523569d-25aa-44a4-97d5-370b93790333 

tier_key.c341ab91-161e-419d-a7e4-6206e75ec097=c341ab91-161e-419d-a7e4-6206e75ec097 

gipsy . tier . instance . 1523569d-25aa-44a4-97d5-370b93790333=8f 5d7614-49e9-4c8a-87f e-c8032a5c9009 

gipsy. connect ion. to. 36485580-5acf-4cl2-9110-47c60e23ddc9=77fd90ec-e49e-4977-a55b-0690e2e0cd08 

gipsy . node . ipaddress . 3=132 . 205 . 8 . 235 

gipsy . node . ipaddress . 2=132 . 205 . 8 . 235 

gipsy .tier . tiertype . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=GMT 

gipsy . tier . name . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=MARFCAT DWT NETO 

gipsy . node . ipaddress . 1=132 . 205 . 8 . 235 

gipsy .tier . name . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=MAIN DST 
gipsy . node . ipaddress . 0=132 . 205 . 8 . 235 

gipsy. t ier. configfile. 89e6605e-5c81-4d95-bf51-8e7576c8dcd7=bin/mult it ier/DSTProfiles/marfcatDWT.config 

gipsy. tier. configfile.77fd90ec-e49e-4977-a55b-0690e2e0cd08=bin/multitier/DSTProfiles/p9.config 

tier_key.77fd90ec-e49e-4977-a55b-0690e2e0cd08=77fd90ec-e49e-4977-a55b-0690e2e0cd08 

gipsy . tier . name . c341ab91-161e-419d-a7e4-6206e75ec097=MARFCAT DGT 

gipsy . instance . name . 8f 5d7614-49e9-4c8a-87f e-c8032a5c9009=MARFCAT 4S0ME 

node_key.3=3 

gipsy .tier . configfile. c341ab91-161e-419d-a7e4-6206e75ec097=/local/data/gipsy/cvscheckout/gipsy/gipsy/bin/multitier/mcirfcatDGT.config 

gipsy . node . ID . 3=3 

node_key.2=2 

gipsy . node . ID . 2=2 

node_key . 1=1 

gipsy . instance . ID . 8f 5d7614-49e9-4c8a-87f e-c8032a5c9009=8f 5d7614-49e9-4c8a-87f e-c8032a5c9009 

node_key.O=0 

gipsy . node . ID . 1=1 

gipsy . node . ID . 0=0 

gipsy . node . color . 3=Lightblue 

gipsy . node . color . 2=Teal 

gipsy . node . color . l=Blue 

gipsy . node . color . 0=Green 

gipsy. tier. name. 1523569d-25aa-44a4-97d5-370b93790333=MARFCAT DWT GLEAM 

gipsy.tier. configfile. 1523569d-25aa-44a4-97d5-370b93790333=/local/data/gipsy/cvscheckout/gipsy/gipsy/bin/mult it ier/marfcatDWT.config 

tier_key.l523569d-25aa-44a4-97d5-370b93790333=1523569d-25aa-44a4-97d5-370b93790333 

gipsy . tier . ID . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6 

gipsy .tier . instance . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=8f 5d7614-49e9-4c8a-87f e-c8032a5c9009 

gipsy .tier . node . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=3 

gipsy. connection. id. 7802414d-437f-4382-9162-15d9d8e735c3=7802414d-437f-4382-9162-15d9d8e735c3 
gipsy . connection . name . 7802414d-437f -4382-9162- 15d9d8e735c3= [MAINGMT , MAIN DST] 
gipsy .tier . posy . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=398 
gipsy .tier . node . c341ab91-161e-419d-a7e4-6206e75ec097=0 

tier_key.29310d89-fd7a-498c-8a47-f80d3b5b9865=29310d89-fd7a-498c-8a47-f80d3b5b9865 

gipsy. connection. from. 6a9el23b-c375-4786-9dl5-12faeee54c59=c341ab91-161e-419d-a7e4-6206e75ec097 

gipsy . tier . ID . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=89e6605e-5c81-4d95-bf 51-8e7576c8dcd7 

gipsy .tier . ID . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=77f d90ec-e49e-4977-a55b-0690e2e0cd08 

gipsy. connection. from. leeea2b5-ec50-4990-a7fe-be48bb5b693e=a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6 

gipsy . tier . posy . c341ab91-161e-419d-a7e4-6206e75ec097=406 

gipsy . connection . id . 0dd7156f -3b40-4df 7-b9a9-dlb44eeb4da0=0dd7156f -3b40-4df 7-b9a9-dlb44eeb4da0 
gipsy . tier . ID . C341ab91-161e-419d-a7e4-6206e75ec097=c341ab91-161e-419d-a7e4-6206e75ec097 
gipsy .tier . node . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=2 
gipsy .tier . node . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=0 
gipsy . t ier . name . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=MAINGMT 

gipsy. connection. from.36485580-5acf-4cl2-9110-47c60e23ddc9=89e6605e-5c81-4d95-bf51-8e7576c8dcd7 

gipsy . tier . configfile . 29310d89-f d7a-498c-8a47-f 80d3b5b9865=/local/data/gipsy/cvscheckout/gipsy/gipsy/bin/multitiermarf cat/GMTNode . conf ig 

gipsy. connect ion. name. 0dd7156f-3b40-4df7-b9a9-dlb44eeb4da0= [MARFCAT DWT GLEAM, MAIN DST] 

connection_key.7802414d-437f-4382-9162-15d9d8e735c3=7802414d-437f-4382-9162-15d9d8e735c3 

gipsy . tier . posy . 89e6605e-5c81-4d95-bf 51-8e7576c8dcd7=224 

gipsy . tier . posy . 77f d90ec-e49e-4977-a55b-0690e2e0cd08=222 

gipsy . tier . isgmt . a409cb6b-ad54-41f 6-af 69-f 4ec78628ec6=f alse 
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